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1.  Introduction 


1.1  Background 

Over  the  last  decades,  the  field  of  neuroscience  has  grown  tremendously,  as  has  our 
understanding  of  how  the  brain  works  in  action.  This  growth  has  prompted  scientists  to  apply 
this  knowledge  to  the  needs  of  the  U.S.  Army  and  find  ways  to  leverage  neuroscientific 
techniques  to  enhance  the  lives  of  our  Soldiers.  For  example,  the  area  of  brain-activity-sensing 
technologies,  such  as  electroencepholography  (EEG),  has  traditionally  been  viewed  as  being 
useful  only  as  a  laboratory  technique  with  limited  operational  application.  However,  recent 
advances  in  the  headset,  electrode,  and  related  data-acquisition  technologies  prove  very 
promising  for  the  transition  to  more  practical  Army  applications  regarding  devices  used  within 
the  U.S.  Department  of  Defense  (DOD)  research  and  development  (R&D)  community,  as  well  as 
increased  plausibility  for  field-deployable  units. 

However,  because  transitioning  EEG  into  operational  settings  remains  a  burgeoning  area  with 
relatively  little  research  history,  it  is  important  to  empirically  evaluate  the  current  state  of  the  art 
in  equipment  and  assess  areas  of  potential  problems  to  see  where  improvements  need  to  be  made 
to  guide  further  development.  In  many  ways,  the  very  metrics  for  evaluating  EEG  systems 
themselves  (in  a  way  that  is  meaningful  for  scientists)  are  not  well  defined  or  standardized  across 
the  R&D  community.  As  a  result,  the  Translational  Neuroscience  Branch  is  endeavoring  to 
facilitate  the  translational  pathways  necessary  for  moving  EEG  into  U.S.  Army  operational 
usage.  A  substantial  component  of  this  translation  is  evaluating  the  efficacy  of  current  EEG 
hardware  solutions. 

EEG  is  commonly  used  to  derive  a  correlation  between  some  type  of  brain  activity  and  a  task  or 
specific  cognitive  event.  In  doing  this,  neuroscientists  examine  EEG  data  within  specific 
temporal  windows,  relative  to  the  onset  or  occurrence  of  an  event  or  series  of  events.  Typically 
data  is  recorded  continuously  while  several  different  events  or  types  of  events  occur,  spread  out 
over  time  (anywhere  from  minutes  to  hours);  data  is  then  binned  into  discrete  sections  of  time 
(epochs)  based  on  when  the  events  occurred,  yielding  an  event-related  response. 

One  caveat  to  this  type  of  approach  is  that  the  timing  of  the  recording  systems  must  be  accurate 
and  precise;  many  neural  responses  of  interest  are  very  fast  and  transient  (on  the  order  of 
milliseconds)  and  may  not  be  properly  captured  and  represented  if  the  recording  system  is 
inconsistent.  That  is,  if  the  stated  times  that  events  occurred  do  not  consistently  represent  the 
actual  time,  then  the  brain  response  to  be  examined  will  not  appear  accurately,  especially  when 
examining  an  average  of  several  events  and  responses.  Thus,  it  is  critical  to  be  aware  of  the 
degree  of  timing  accuracy/consistency  of  any  system  to  be  used  in  this  manner. 
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1.2  Jitter  and  Drift 


This  report  outlines  two  specific  areas  of  potential  concern  related  to  event  timing  within 
contemporary  EEG  systems:  encoding  jitter  and  temporal  drift.  Encoding  jitter  refers  to 
variability  between  the  time  that  an  event  is  stated  or  reported  to  have  occurred  and  when  it 
actually  occurs.  Within  an  EEG  data  stream,  there  are  at  least  two  primary  sources  where  jitter 
can  occur.  The  first  stems  from  inconsistency  or  unreliability  of  the  device  that  creates  the 
stimulus  or  event  to  which  the  participant  is  responding.  That  is,  there  is  some  difference  in  time 
between  when  the  device  is  said  to  have  begun  creating  the  stimulus  (such  as  a  function  call 
within  a  computer  script)  and  when  the  physical  stimulus  arrives  (such  as  a  sound  wave  coming 
out  of  a  speaker).  These  types  of  inconsistencies,  often  described  as  stimulus-related  jitter, 
usually  originate  from  devices  that  are  not  built  for  the  timing  specifications  of  the  task  at  hand 
or  perhaps  from  asynchronous  operating  frequencies  between  devices  (such  as  a  CPU  call  to 
draw  a  picture  on  a  monitor  that  refreshes  at  only  60  Hz).  The  second,  recording-related  jitter, 
occurs  when  there  is  variability  or  asynchrony  in  the  system  that  is  recording  the  occurrence  of 
events  and  tying  this  to  the  EEG  data  stream.  In  most  conventional  EEG  systems,  this  would  be 
related  to  the  accuracy  or  sample  rate  of  the  digital-to-analog  converter  (DAC)  device  or 
software  routines  (such  as  buffering)  used  in  the  recording  system  to  integrate  the  signals.  In  this 
report,  we  are  primarily  targeting  this  source  of  jitter  as  it  relates  to  specific  recording  systems. 

Temporal  drift  occurs  when  there  is  a  slow,  cumulative  difference  in  time  coefficients  of  two  or 
more  systems.  It  is  primarily  related  to  having  two  independent  clocks  that  may  not  be  exactly 
100%  synchronous  with  one  another.  When  clocks  are  out  of  synchrony,  the  times  that  events 
occur  relative  to  one  clock  will  “drift  away”  from  reports  of  the  other.  In  EEG,  a  PC  is  often 
used  to  provide  stimuli  to  a  subject  (e.g.,  play  sounds,  present  video)  which  logs  events  based 
upon  that  PC’s  internal  clock.  Meanwhile,  the  EEG  system  itself  typically  has  its  own  clock 
(based  on  the  DAC)  that  is  of  a  different  (lower)  sampling  rate  and  in  many  cases  is  run  off  of  a 
second  PC.  “Time”  relative  to  the  EEG  is  then  converted  from  its  own  sample  rate  to  seconds  or 
milliseconds,  based  on  an  assumed  sample  rate  (e.g.,  512  Hz);  however,  if  the  actual  rate  is 
slightly  different  (e.g.,  512.001  Hz),  then  an  asynchrony  is  introduced. 

This  drift  is  commonly  alleviated  by  using  synchronizing  data  triggers.  Virtually  all  modem  EEG 
systems  have  some  means  of  receiving  information  from  external  systems;  that  information  is 
integrated  with  the  EEG  data  stream  online  (see  section  2  for  several  examples).  This  may  be 
done  through  communication  over  a  parallel,  serial,  or  network  port  (if  using  multiple  PCs), 
Transmission  Control  Protocol  (TCP),  or  direct  function  calls  to  pass  data  between  stimulus  and 
recording  programs  within  the  PC  collecting  data.  By  doing  so,  data  representing  the  timing  of 
external  events  is  added  to  the  EEG  data,  sampled  within  the  same  time  and  rate,  and  thus  is 
natively  in-synchrony  with  it. 

However,  situations  commonly  arise  where  data  originally  recorded  online  are  not  sufficient  for 
current  needs,  and  information  must  be  gathered  from  a  second  log  file.  For  example,  an 


2 


experimenter  may  realize  during  data  analysis  that  he  or  she  would  like  to  examine  evoked 
potentials  related  to  the  exact  time  at  which  the  participants  pressed  a  button  in  response  to  a 
scene  flashed  on  a  screen.  Although  the  response  times  were  not  embedded  with  the  EEG  during 
the  data  collection,  the  experimenter  has  a  detailed  log  of  the  events  and  subjects’  responses, 
which  were  created  by  the  program  running  the  experiment.  The  experimenter  could  easily 
extract  the  necessary  data  from  this  second  log  file;  however,  because  the  response  times  were 
logged  using  a  different  clock  system,  they  are  not  necessarily  on  the  same  time  scale  and  may 
be  out  of  synchrony  with  the  original  EEG.  If  the  data  were  then  binned  based  on  those 
externally  encoded  times,  it  could  introduce  a  progressive  error  when  comparing  across  many 
trials. 


1.3  Objectives 

There  are  three  primary  objectives  for  this  report.  First,  we  aim  to  assess  the  typical  encoding 
jitter  and  timing  drift  observed  within  four  different  commercially  available  EEG  systems — three 
contemporary  wireless  systems  (manufactured  by  Advanced  Brain  Monitoring,  Emotiv  Systems, 
and  Quasar  Inc.)  and  one  traditional  wire -based  system  (Biosemi).  Second,  we  highlight  the 
effects  on  both  real,  empirically  derived,  and  simulated  EEG  data,  and  the  consequences  if  these 
factors  are  not  appropriately  accounted  for  and  drift  occurs  in  the  data.  Finally,  we  discuss 
methods  that  can  correct  timing  inconsistencies  and  show  the  efficacy  of  those  methods  for  not 
only  correcting  drift,  but  also  potentially  improving  the  timing  fidelity  beyond  that  of  the  initial 
recording. 


2.  Methods 


2.1  Hardware  Configuration 

The  overall  configuration  consisted  of  two  PCs  plus  the  EEG  systems  being  tested.  The  primary 
EEG  data  acquisition  (“DAQ”)  PC  was  a  Dell  Precision  690  running  Windows  XP  SP3, 
connected  to  the  following  EEG  systems  (with  their  respective  acquisition  software  versions): 
Advanced  Brain  Monitoring  X10  (Be-Alert  2.45*),  Biosemi  Active2  (ActiView  6.05),  Emotiv 
Epoc  (TestBench  1.5.0. 3),  and  Quasar  Helmet-Mounted  System  (HMS)  (QStreamer  1.5.1046). 
All  systems  were  connected  to  the  acquisition  PC  through  a  USB  hub.  The  stimulus-trigger¬ 
generating  (“STIM”)  PC  was  a  Dell  Optiplex  GX620  PC,  also  running  Windows  XP  SP3.  For 
Advanced  Brain  Monitoring  (ABM)  and  Biosemi  systems,  this  STIM  PC  was  connected  directly 
to  the  EEG  systems  using  a  25 -pin  parallel  port  into  an  integration  box  that  was  part  of  each  EEG 
DAC  system.  For  Quasar  HMS,  this  parallel  port  signal  was  converted  to  a  single  transistor- 


In  response  to  feedback  during  testing,  Advanced  Brain  Monitoring  provided  a  firmware  update  that  improved  the  timing 
drift  observed  in  their  DAC  unit.  We  include  here  results  from  both  the  original  and  updated  versions  (referred  to  as  "ABM -A” 
and  "ABM-B”)  for  additional  comparison. 
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transistor  logic  (TTL)  binary  pulse,  as  that  is  the  only  available  method  of  trigger  input  for  that 
system.  The  Emotiv  Epoc  system  used  a  serial  connection  (RS-232,  57,600  bps)  between  the 
two  PCs,  relying  on  the  DAQ  PC  and  acquisition  software  for  online  integration.  E-Prime  2.0 
(Psychology  Software  Tools,  Inc.)  was  used  to  send  stimulus  trigger  pulses  (during  testing)  and 
record  pulse  receipt  times  (during  calibration  verification).  This  general  configuration  (two 
separate  PCs)  was  chosen  because  it  is  a  common  scenario  for  behavioral  research  labs,  and  it  is 
the  most  likely  case  for  temporal  drift  and  jitter  artifacts  to  occur.  During  all  tests,  only  one  EEG 
system  was  active  at  a  time. 

2.2  Calibration 

The  primary  factors  of  interest  in  this  report  are  the  encoding  jitter  and  temporal  drift  observed  in 
several  commercially  available  EEG  data  acquisition  systems.  Encoding  jitter  in  this  case  is 
defined  as  variability  (in  +/-  millisecond  standard  deviations)  in  the  reported  time  of  arrival  of  a 
signal,  relative  to  a  known-quantity  time  at  which  it  actually  occurred.  Temporal  drift  refers  to 
an  increasing  difference  over  time  between  the  reported  time  of  arrival  and  the  known  time  at 
which  it  occurred,  and  is  reported  as  a  fractional  rate. 

In  order  to  quantify  both  of  these  factors,  they  must  be  compared  against  a  known-quantity, 
consistent,  standard  signal.  To  establish  the  reliability  of  our  standard  signal,  we  first  used  a 
calibrated  function  generator  (Tektronix  AFG320)  to  send  a  1-Hz  a  square  wave  to  the  DAQ  PC 
using  a  BNC-to-DB25  adapter  to  connect  to  the  PC’s  parallel  port,  simulating  a  TTL-level 
signal.  The  time  of  arrival  of  the  leading  edge  of  each  waveform  was  recorded  using  E-Prime; 
these  times  were  confirmed  to  line  up  consistently  within  +/-0.3  ms  with  no  notable  drift  over  a 
10-min  recording  window.  This  confirms  that  the  parallel-port/E- Prime  combination  can  reliably 
(within  <l-ms  precision)  report  pulse  times,  and  that  the  DAQ  PC  clock  does  not  drift  relative  to 
a  calibrated  source.  As  a  second  step,  parallel-port  triggers  were  sent  (10-ms  pulse,  every  100 
ms)  using  E-Prime  from  the  STIM  PC  and  recorded  using  E-Prime  on  the  DAQ  PC.  These  were 
confirmed  to  be  within  +/-0.3  ms  variance  in  time  of  arrival  (jitter)  and  showed  no  notable 
increase  in  difference  (drift)  over  the  10-min  record,  affirming  that  the  output  of  the  STIM  PC 
using  E-Prime  is  functionally  equivalent  to  that  of  the  calibrated  function  generator  and  the 
system  clocks  of  the  two  PCs  did  not  drift  relative  to  one  another  within  a  typical  EEG  recording 
timescale. 

2.3  Timing  Test  Paradigm 

To  test  the  consistency  and  drift  in  recorded  times  for  each  EEG  system,  a  digital  value  code  was 
sent  from  the  STIM  PC  every  1250  ms  and  recorded  using  the  native  acquisition  software  for 
each  EEG  system.  This  timeframe  was  chosen  because  it  is  not  evenly  divisible  by  the  native 
sample  rate  of  any  of  the  systems  (ABM:  256  Hz;  Biosemi:  512  Hz;  Emotiv:  128  Hz;  Quasar: 

240  Hz)  and  thus  would  avoid  any  aliasing  effects  and  maximize  the  average  number  of  cases 
where  the  trigger  arrival  time  fell  between  sample  periods  across  systems.  This  is  also  within  the 
range  of  a  typical  time  between  stimuli  or  trials  for  many  behavioral  tasks.  Five  hundred  triggers 
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were  sent,  covering  a  total  of  10.42  min,  as  this  ensured  a  large  sample  size  as  well  as  fairly 
lengthy  time  that  a  participant  might  perform  a  typical  event-related  task.  No  EEG  data  were 
recorded  during  these  times,  as  only  the  trigger  timing  was  of  interest. 

2.4  Waveform  Simulation  With  Phantom 

In  order  to  highlight  the  functional  effects  of  timing  jitter  and  drift  on  the  estimation  of  an  EEG 
signal,  it  was  necessary  to  use  a  known-quantity  signal  for  comparison  that  could  be  detected 
using  a  standard  EEG  system.  This  was  done  using  an  EEG  “phantom”  head  prototype  device 
created  by  Creare,  Inc.,  in  conjunction  with  Army  Small  Business  Innovative  Research  Award 
no.  A 10-066.  The  device  is  a  mannequin-style  head  shape  made  of  multiple  conductive  layers 
approximately  analogous  to  the  human  brain,  skull,  and  skin  in  both  physical  shape  and  electrical 
conductance,  containing  several  open-ended  electrode  leads  encapsulated  within  the  “brain” 
material.  Using  the  calibrated  function  generator,  we  created  a  single  10-Hz  sine  wave  that  was 
passed  through  an  electrode  located  approximately  within  the  frontal  cortex  region  of  the  device. 
The  device  was  triggered  at  regular  intervals  in  order  to  simulate  a  “response”  that  could  be 
measured  at  the  scalp  analogous  to  a  typical  event-related  potential  (ERP). 

2.5  Simulated  Data 

To  highlight  the  functional  effects  of  system  variance  and  drift  on  a  continuous  signal,  the  error 
was  simulated  by  application  to  a  square  waveform.  The  initial  (basis  function)  waveform  rose 
from  0  to  1  at  time  100  ms,  returning  to  0  after  a  100-ms  period.  This  was  repeated  200  times 
with  4-s  inter-stimulus-interval  (ISI)  for  a  total  length  of  13.33  min,  to  simulate  a  typical 
experimental  trial  length  interval  and  typical  total  time  of  a  single  test.  Effects  of  drift,  jitter,  and 
total  recording  time  were  simulated  by  varying  each  of  these  factors  in  the  original  log  of  event 
times  prior  to  breaking  the  data  into  “epochs.” 

2.6  Visual-Evoked  Potential  Paradigm 

Data  are  also  discussed  using  the  EEG  waveforms  elicited  from  a  visual-evoked  potential  (VEP) 
paradigm,  collected  as  part  of  an  ongoing  human  research  study.  This  task  was  chosen  because  it 
elicits  a  very  well-characterized,  stereotypical  EEG  response  that  is  time  locked  to  the 
occurrence  of  events.  In  this  task,  participants  (three  males,  five  females)  sat  comfortably  in 
front  of  a  computer  monitor.  A  black  fixation  cross  is  centered  on  the  screen  and  remains  on 
through  all  presentations.  On  each  trial,  a  visual  stimulus  appears  in  the  center  of  the  screen  for 
150  ms  followed  by  1640-  to  2155-ms  interstimulus  intervals  containing  only  the  fixation  cross. 
On  12%  of  the  trials,  the  stimulus  is  a  picture  of  an  insurgent  (rare),  and  on  the  other  88%,  the 
stimulus  is  a  U.S.  Soldier  (standard),  both  presented  against  a  white  background.  Participants  are 
instructed  to  press  a  response  key  after  each  stimulus.  VEPs  described  here  were  calculated  as 
the  mean  signal  from  all  “standard”  (U.S.  Soldier)  trials,  after  standard  preprocessing  (filtering, 
artifact  rejection,  baseline  correction). 
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2.7  Correction  Algorithm 

We  tested  an  algorithm  that  was  developed  to  address  both  clock  drift  and  stimulus  timing  jitter 
and  designed  to  be  relatively  universal  for  any  EEG  system.  It  requires  only  that  the  system  is 
able  to  receive  a  single  binary  pulse  trigger  input  (as  with  the  Quasar  system)  and  a  minimum 
trigger  encoding  accuracy  of  approximately  +1-15  ms.  The  process  relies  on  using  a  secondary 
log  file  generated  by  the  STEM  PC  of  all  stimulus  events;  this  log  file  ultimately  contains  the 
onset  data  of  interest  for  a  research  study.  The  process  contains  two  primary  parts: 

•  Synchronization  Phase:  At  the  beginning  of  EEG  data  collection,  the  STIM  PC  sends  a 
series  of  codes  to  the  EEG  system,  consisting  of  the  same  value  repeated  3 1  times,  with 
alternated  spacing  of  200  and  300  ms  between  triggers.  The  onset  of  each  pulse  is  recorded 
in  a  log  file  on  the  STIM  PC  as  well  as  by  the  EEG  system  controlled  by  the  DAQ  PC. 

•  Alignment  Phase :  After  data  collection,  the  onset  times  of  the  31  codes  recorded  in  the 
STEM  log  file  are  used  as  a  predictor  for  the  corresponding  values  in  the  online-encoded 
ACQ  PC  log  file  using  linear  regression.  The  intercept  of  the  regression  line  is  used  to 
align  the  first  event  of  the  STEM  PC  log  file,  so  that  it  can  replace  the  original  encoded 
(potentially  high-jitter)  values.  Additionally,  prior  to  replacement,  the  new  onset  times  are 
multipled  by  a  known  “drift  factor,”  which  relates  the  two  clocks  of  the  STIM  PC  and  EEG 
system;  this  factor  can  be  derived  from  either  the  slope  of  the  regression  line  calculated 
here  or  prior  testing  with  longer-range  time  vectors  (the  method  chosen  for  this  report). 


3.  Results  and  Discussion 


3.1  Drift  and  Jitter  Observed 

For  each  EEG  system,  drift  was  calculated  by  comparing  the  trigger  event  times  reported  by  the 
system  with  those  reported  by  the  E-Prime  logfile  (from  the  STIM  PC).  SpecificaEy,  linear 
regression  was  used  to  calculate  the  slope  of  the  best-fit  lines  between  the  two  sets  of  times, 
using  the  E-Prime  logfile  as  a  predictor  of  the  EEG  system  time.  This  estimates,  on  average, 
how  much  one  clock  drifts  relative  to  the  other,  per  unit  time.  Additionally,  we  calculated  the 
mean  and  standard  deviation  of  the  difference  in  reported  trigger  times  across  the  500  events  and 
estimated  the  average  variability  (jitter)  related  to  the  trigger  encoding  system  (table  1).  Across 
the  four  systems  tested,  a  wide  range  of  drift  and  variance  was  observed.  Quasar’s  HMS  headset 
showed  almost  no  measurable  drift,  while  that  of  the  Emotiv  Epoc  exceeded  0.1%.  A  comparison 
of  the  two  ABM  measurements  (representing  before  and  after  a  firmware  update)  showed  nearly 
a  10-fold  difference  in  drift,  and  two  times  the  improved  variability,  marking  a  sizeable 
improvement  in  precision.  While  the  overaE  trial-to-trial  variabihty  was  relatively  smaE 
(<1  ms),  the  exception  to  this  was  the  Emotiv  Epoc,  which  reported  event  times  that  varied 
substantiaEy  (by  several  sample  periods)  from  our  verified  timing  on  almost  every  trial. 
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Table  1.  Timing  drift  and  mean  error  for  each  EEG  system 


System  Mfr 

Drift 

(%) 

Mean  Error 

(ms) 

Std.  Dev.  Error 

(+/-  ms) 

Sample  Rate 

Acq.  S/W 

ABM:  A 

0.027575 

0.344689 

0.479946 

256 

B -Alert  2.45 

ABM:  B 

0.002565 

0.032064 

0.281587 

256 

B -Alert  2.45 

Biosemi 

0.001879 

0.023484 

0.109064 

512 

ActiView  6.05 

Emotiv 

0.106463 

1.330787 

55.705045 

128 

TestBench  1.5. 0.3 

Quasar 

-0.000626 

-0.007828 

0.123526 

240 

Qstreamer  1.5.1046 

3.2  Simulated  Effects  of  Drift 

Given  the  extent  and  range  of  drift  and  jitter  observed,  we  sought  to  investigate  the  potential 
effects  of  these  issues  if  not  properly  accounted  for  and  the  extent  to  which  they  may  pose  a  true 
pragmatic  problem  for  evoked-potential  data.  As  an  initial  step,  figure  1  shows  the  simulated 
effect  of  a  range  of  hypothetical  timing  drift  values,  as  well  as  those  observed  in  these  systems, 
applied  to  a  simple  square  wave  function. 


0  100  200  300  400  500 

Time  (ms) 


Figure  1.  The  simulated  effect  of  a  range  of  hypothetical  timing  drift  values  (dashed  lines),  as  well  as 
those  observed  in  these  systems  (solid  lines),  applied  to  a  simple  square  wave  function 
consisting  of  125  trials  over  8.33  min. 

This  function  represents  a  simulation  of  125  “trials”  with  4  s  ISI,  approximating  an  experimental 
recording  session  of  8.33  min,  common  for  many  behavioral  experiments.  As  seen  here,  the 
overall  effect  is  that  of  widening  the  base  of  the  function  while  distorting  the  slope  and 
suppressing  the  overall  peak;  the  effect  observed  with  the  drift  measured  in  the  original  ABM 
firmware  (ABM:  A,  0.0278)  is  a  12%  decrease  in  amplitude  and  near  doubling  of  base  width, 
while  the  peak  with  Emotiv-simulated  drift  is  suppressed  by  >76%. 
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Because  this  effect  increases  the  difference  in  stated  records  over  time  (e.g.,  a  progressive  error), 
the  extent  of  the  effect  increases  as  the  length  of  recording  increases.  In  order  to  depict  the 
extent  of  this  effect  on  typical  recording  times,  figure  2  shows  the  effect  of  different  total  record 
lengths  for  two  observed  drift  amounts:  that  observed  with  the  ABM  system  originally  (figure 
2a)  and  after  the  provided  firmware  update  (figure  2b). 


Estimated  signal  change  per 
recording  length,  0.0275%  drift 


Estimated  signal  change  per 
recording  length,  0.0025%  drift 
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Figure  2.  The  effect  of  different  total  record  lengths  for  two  observed 
drift  amounts  from  the  two  different  ABM  firmware 
versions. 
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With  the  larger  amount  of  drift,  a  notable  effect  on  the  signal  waveform  occurs  within  only  a  few 
minutes  of  recording  time.  Meanwhile,  if  the  drift  is  much  smaller,  the  overall  effect  is  relatively 
minimal. 

While  these  data  show  the  potential  effect  of  drift,  they  are  based  on  a  static  image  with  no 
intertrial  variance  or  realism  for  human  applications.  Figure  3  shows  scalp  voltage  event-related 
potential  measured  at  channel  Oz  using  the  Biosemi  system  and  confirmed  correct,  time-locked 
trigger  event  codes.  As  in  figure  1,  results  are  plotted  using  data  with  varied  amounts  of  drift 
applied,  including  those  measured  in  the  EEG  systems. 


none 

0.00077  (Quasar) 
0.001 

0.00170  (Biosemi) 
0.00253  (ABM:B) 
0.01 
0.02 

0.02731  (ABM  :A) 

0.03 

0.04 

0.05 

0.06 

0.07 

0.08 

0.08783  (Emotiv) 

0.09 

0.1 


Signal  change  per  %  drift,  ERP 


Figure  3.  The  simulated  effect  of  timing  drift  applied  to  scalp  voltage  event -related  potential  measured 
at  channel  Oz  using  the  Biosemi  system  and  confirmed  correct,  time-locked  trigger  event 
codes.  Data  represent  the  response  from  a  VEP  task. 

While  the  signal  is  only  minimally  distorted  from  the  drift  values  observed  with  the  Quasar, 
Biosemi,  and  updated  ABM  systems,  application  of  a  0.01%  drift  dramatically  attenuates  the 
ERP  peak,  increases  the  latency,  and  widens  the  response  slope.  Meanwhile,  this  predicts 
substantial  degradation  if  data  from  the  original  ABM  firmware  (ABM:  A)  or  Emotiv  do  not 
account  for  drift,  even  to  the  extent  that  an  ERP  would  be  unrecognizable. 

3.3  Simulated  Effects  of  Jitter 

The  amount  of  jitter,  measured  here  as  variance  in  the  difference  between  system-encoded  and 
actual  stimulus  onset  times,  can  also  affect  the  clarity  of  a  recorded  signal.  For  three  of  the  four 
systems  investigated  here,  variability  was  relatively  small — less  than  +/-1  ms  around  the  mean 
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signal.  Simulations  of  these  amounts  of  jitter  showed  no  noticeable  effect  on  simulated  or  real 
ERP  waveforms  and  are  well  below  the  scope  of  typical  EEG  systems. 

In  contrast,  the  Emotiv  Epoc  system  displayed  tremendous  variance  in  the  reported  event  times. 
Figure  4  shows  the  distribution  of  differences  between  actual  and  reported  event  times. 


Trigger  Error  in  Emotiv  Epoc 

30 


Difference  between  Actual  and  Reported 
Onset  Time  (ms) 


Figure  4.  Distribution  of  recording  errors  (difference  between  actual 
and  reported  trigger  onset  time)  measured  with  the  Emotiv 
Epoc  EEG  headset. 

Surprisingly,  the  system-reported  times  were  never  fully  correct  (e.g.,  0-ms  difference),  but 
rather  always  either  ahead  or  behind  by  several  samples  (1  sample  length  =  ~7.3  ms)  resulting  in 
a  bimodal  distribution.  To  highlight  how  this  extreme  jitter  likely  affects  recorded  signal  quality, 
figure  5  shows  the  effects  of  applying  the  difference  values  recorded  onto  the  same  square  wave 
(figure  5a)  and  real  human-derived  ERP  (5b)  described  previously. 

That  is,  they  illustrate  how  one  might  expect  the  signal  derived  from  the  Epoc-encode  d  stimulus 
timing  to  be  affected  solely  based  on  the  variability  of  that  component  of  the  system.  Note  that 
for  a  perfect  signal,  the  amplitude  is  dramatically  decreased  and  width  nearly  doubled,  while  for 
more  realistic  data  (from  human  ERPs),  the  signal  is  nearly  completely  attenuated. 
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Simulated  effect  of  stimulus  timing 
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Figure  5.  Simulation  of  the  anticipated  effects  of  the  variance  by 
applying  the  difference  values  recorded  onto  the  same 
square  wave  (A)  and  real  human-derived  ERP  (B). 
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3.4  Observations  in  Real  Data  (Phantom) 


The  results  just  outlined  suggest  that  data  recorded  from  the  Emotiv  Epoc  system  may  be 
dramatically  impacted  by  such  large  trigger  encoding  jitter.  However,  since  these  are  based 
solely  on  simulating  a  single  parameter,  it  is  unclear  whether  this  simulation  truly  encapsulates 
the  real-world  impact  of  data  variance  and  drift.  In  order  to  fully  estimate  the  overall  quality  of 
data  that  might  be  expected  from  this  unit,  we  performed  an  additional  session  using  a 
“phantom”  head-shaped  device.  This  device  provides  the  opportunity  to  simulate  the  recording 
and  processing  of  data  just  as  from  human  subjects  but  with  the  advantage  of  providing  a  known- 
quantity  original  signal  for  the  basis  of  comparison;  this  would  not  be  possible  from  true  human 
data  or  postrecording  simulation. 

Figure  6  shows  data  recorded  using  the  Emotiv  Epoc  system  in  conjunction  with  the  phantom 
head  device.  The  input  signal,  a  single  10-Hz  sine  wave,  was  programmed  to  occur  in  response 
to  the  stimulus  paradigm  for  the  VEP  described  previously.  That  is,  a  single  10-Hz  wave  was 
initiated  at  the  onset  of  every  trial,  akin  to  an  event -related  response.  Data  were  epoched  based 
on  external  triggers  encoded  online  using  the  vendor-provided  software  (TestBench  1.5.03)  and 
processed  identically  to  human-derived  data  described  previously. 
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Figure  6.  Data  recorded  using  the  Emotiv  Epoc  system  in  conjunction 
with  a  phantom  head  device.  Natively  reported  data  are 
shown  in  red. 


12 


Note  that  the  recorded  signal  (solid  red)  is  distorted,  attenuated,  and  extended  relative  to  the 
input  (black,  dashed)  signal,  most  likely  the  effect  of  the  varied  timing.  A  fixed  delay  of  just 
under  100  ms  in  the  initial  response  was  also  observed. 

3.5  Correcting  for  Timing  Error 

In  order  to  compensate  for  the  timing  error  described  here  (excessive  encoding  jitter  and 
potential  drift  from  referring  to  a  secondary  timing  log),  we  have  developed  an  algorithm  that 
directly  addresses  both  of  these  issues.  The  goal  of  this  process  is  to  draw  on  records  from  a 
second  device  with  higher  timing  accuracy  than  the  EEG  system  in  question  while  ensuring  that 
no  drift-related  error  is  introduced. 

Figure  7  shows  the  results  of  applying  this  algorithm  to  the  data  from  the  Emotiv  system 
recorded  on  the  phantom  device.  Unlike  the  data  derived  from  the  natively  encoded  trigger  times 
(red  line,  figure  6),  the  corrected  signal  (solid  green)  now  closely  resembles  the  original  10-Hz 
input  function,  without  substantial  distortion  or  widening  of  the  overall  form  of  the  function. 

The  additional  negative  deflection  in  the  derived  signal  is  likely  an  effect  from  the  headset’s 
automatic  filtering. 
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Figure  7.  Results  of  applying  a  correction  algorithm  to  the  data  from 
the  Emotiv  system. 
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The  procedure  was  also  applied  to  human  VEP  responses  from  the  ABM  (figure  8a)  and  Emotiv 
(5b)  systems.  Blue  lines  represent  the  original  “native”  waveform,  red  depict  an  example  of 
imported  data  without  adjusting  for  drift,  and  green  show  results  using  the  compensation 
algorithm. 
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Figure  8.  The  same  correction  procedure  applied  to  human  VEP 
responses  from  the  ABM  (A)  and  Emotiv  (B)  systems. 


Note  that  for  ABM,  which  has  relatively  high  timing  precision  and  minimal  drift,  there  is  little 
need  for,  or  effect  of,  correction.  The  signal  based  on  the  Emotiv  system,  in  contrast,  which  has 
a  substantially  higher  drift  and  variability  rate,  is  noticeably  improved  using  the  procedure, 
especially  within  the  early  fast- phase  response  components  (circled). 
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4.  Discussion 


Here,  we  have  quantified  the  amount  of  drift  and  stimulus-encoding  jitter  that  occurs  within 
several  contemporary,  commercially  available  EEG  systems,  and  shown  the  implications  for  the 
analysis  of  typical  event-related  potentials.  We  have  found  a  wide  range  of  both  drift  and 
encoding  jitter,  running  the  gambit  from  barely  measurable  (and  likely  trivial,  e.g.,  Quasar  or 
Biosemi)  to  fairly  substantial  (e.g.,  Emotiv  and  ABM’s  early  firmware).  Overall,  we  have  shown 
that  if  timing  drift  is  not  accounted  for,  the  effect  on  data  averaging  could  be  potentially 
problematic.  This  concern  increases  with  longer  recording  times,  especially  those  going  beyond 
8-10  min  on  a  single  record.  The  extent  of  this  problem  relies  heavily  on  the  total  average  drift 
of  the  system;  when  the  drift  is  small,  the  potential  effect  is  minimal,  unless  the  recording  time  is 
excessive  (e.g.,  >30  min).  However,  data  derived  from  systems  showing  -0.01%  drift  or  more 
are  altered  catastrophically  if  this  is  not  accounted  for  and  thus  must  be  of  concern  to  users. 

We  include  data  here  from  one  system  (ABM’s  X-10  unit)  using  two  different  firmware 
versions:  one  originally  supplied  with  the  unit  and  an  updated  version  supplied  to  us  by  the 
vendor.  In  this  case,  a  near  10-fold  improvement  was  measured  in  the  overall  drift,  and  2-fold 
improvement  in  encoding  jitter.  Note  that  this  (as  highlighted  in  figures  1-3)  had  a  fairly 
dramatic  impact  on  the  ability  to  properly  estimate  a  native  signal,  due  only  to  a  fairly  simple 
software  patch.  This  improvement  arose  directly  from  feedback  and  discussion  with  ABM 
personnel,  highlighting  the  benefits  gained  from  vendor/lab  interaction. 

Of  the  four  systems  evaluated  here,  three  are  “wireless”  (meaning  no  physical  tether  between  the 
subject  and  DAQ  PC)  and  newly  (within  recent  years)  released  systems.  In  contrast,  the  Biosemi 
ActivTwo  system,  which  relies  on  a  physical  connection  to  the  recording  PC,  has  been  in  wide 
use  for  several  years  and  is  often  considered  the  “gold  standard”  of  laboratory  EEG.  While  the 
use  of  wireless  transmission  may  raise  the  question  of  additional  synchronization  problems  (such 
as  from  dropped  packets  or  temporary  data  loss),  no  evidence  of  decreased  performance  in  the 
realm  of  timing  drift  and  jitter  was  seen  here.  In  fact,  the  lowest  drift  observed  occurred  with  the 
Quasar  unit,  which  had  timing  variance  comparable  to  Biosemi’s  system. 

The  Emotiv  Epoc  system  showed  significantly  more  drift  and  encoding  jitter  than  the  other 
systems  tested.  While  this  causes  concern,  a  few  caveats  should  be  noted  related  to  these 
inaccuracies.  First,  all  other  systems  tested  here  (as  well  as  most  conventional  EEG  systems)  use 
a  direct  input  into  the  EEG  system’s  DAC,  typically  coming  from  the  STIM  PC  parallel  port. 

The  Epoc,  in  contrast,  has  no  external  input,  and  for  third-party  external  triggering  relies  on  input 
directly  to  the  ACQ  PC  controlling  the  device  and  thus  must  be  handled  within  software  on  that 
PC  instead.  Additionally,  TestBench  (the  software  provided  by  the  vendor)  provides  only  input 
via  serial  port.  While  the  data  transmission  rate  is  very  fast  (recorded  here  at  56  kbps), 
communication  access  is  only  through  the  Windows  buffer,  which  creates  delays  and 
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inaccuracies.  Thus,  it  is  very  possible  that  the  extreme  jitter  observed  here  arises  simply  from 
the  trigger  acquisition  method  and  is  not  native  to  the  device  itself.  Second,  the  retail  cost  of  the 
Emotiv  Epoc  is  currently  only  $700  (for  the  research  edition),  a  small  fraction  of  the  cost  for  any 
of  the  other  systems  (which  are  in  the  tens  of  thousands  of  dollars).  Given  this  price  point,  it  is 
unclear  whether  this  unit,  which  is  targeted  for  the  consumer  market,  should  even  be  expected  to 
perform  comparably.  Finally,  the  sales  team  at  Emotiv  Systems  is  very  clear  that  their  target 
application  does  not  focus  on  averaged  evoked  potentials.  Nevertheless,  we  have  been  in 
continual  contact  with  their  technical  team,  who  are  aware  of  the  degree  of  our  results, 
acknowledged  the  problem,  and  requested  continual  feedback.  Currently,  they  have  provided  a 
Beta  version  of  TestBench,  which  we  have  tested,  and  observed  an  approximate  50%  decrease  in 
overall  variance  and  drift.  However,  since  this  version  has  not  been  officially  released  to  the 
public  as  of  the  time  of  this  report,  we  did  not  feel  it  was  appropriate  for  the  review  published 
here.  Based  on  our  interactions,  we  are  hopeful  of  improved  timing  in  the  near  future. 

Regardless  of  these  concerns,  however,  we  have  also  shown  that  using  an  additional 
synchronization  and  compensation  algorithm  can  overcome  these  errors  and  provide  a  fairly 
robust  signal. 


5.  Conclusions 


In  summary,  we  have  shown  that  a  wide  range  of  timing  drift  and  variance  occurs  across 
different  EEG  systems  relative  to  a  standard  system  clock.  Depending  on  the  extent  of  drift  that 
occurs,  it  could  be  potentially  catastrophic  for  averaged  evoked  potential  analyses  and  thus  must 
be  monitored  carefully.  In  cases  where  secondary  data  files  must  be  referenced,  lab  personnel 
should  consider  using  an  additional  synchronization  algorithm,  which  we  have  shown  to  properly 
compensate  for  drift-related  errors.  In  addition,  use  of  such  an  algorithm  can  even  improve  the 
data  quality  from  a  system  that  may  be  otherwise  questionable. 


6.  Recommendations 


The  detrimental  effects  of  timing  drift  discussed  here  are  a  potential  problem  only  under  certain 
circumstances — specifically,  when  a  second  device  is  used  to  provide  a  log  of  event  times.  An 
example  would  be  when  a  researcher  refers  to  a  log  file  created  by  the  PC  that  provides  stimuli  to 
a  subject,  or  logged  data  by  an  additional  device  such  as  an  eye  tracker,  electrocardiogram 
device,  or  motion  capture  system.  However,  if  data  analysis  relies  solely  on  trigger  information 
recorded  natively  by  the  EEG  system  online,  then  signal  drift  will  not  be  a  concern. 

Additionally,  as  shown  here,  it  is  only  a  concern  if  either  the  native  drift  of  the  system  is  sizably 
large  (e.g.,  >0.01%)  or  the  data  records  being  analyzed  are  extremely  lengthy  in  time.  That  is,  it 
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would  not  likely  be  a  concern  with  Quasar  HMS-system  derived  data,  as  it  has  natively  almost 
no  measurable  drift. 

Our  primary  recommendation  is  that  researchers  should  quantify  the  drift  expected  for  the 
system  they  use,  similar  to  the  methods  described  here.  This  will  serve  two  purposes.  First,  it 
will  identify  the  degree  to  which  this  issue  must  be  addressed  in  the  future.  Second,  the  data  will 
provide  the  researcher  with  a  quantified  “drift  factor,”  which  can  be  used  for  correction 
algorithms  later  if  necessary. 

For  those  concerned  about  ensuring  that  drift  does  not  adversely  affect  average  ERP  analyses,  we 
recommend  choosing  among  the  following  approaches: 

1.  Rely  Solely  on  Embedded  Triggers.  Trigger  data  encoded  by  the  EEG  system  online 
received  from  a  secondary  source  are  very  unlikely  to  show  drift.  This  is  accomplished 
either  through  a  direct  cable  connection  (if  using  a  second  PC)  or  online  software 
communication,  such  as  TCP  protocols.  However,  it  is  not  always  possible  to  predict  all  of 
the  event  information  that  may  be  desired  to  align  with  the  EEG  prior  to  beginning  a  study. 
Additionally,  hardware  limitations  may  limit  what  can  be  embedded  during  acquisition. 

For  example,  the  Quasar  HMS  system  provides  only  a  single  binary  TTL  input  for  external 
triggers.  As  a  result,  the  native  data  contains  only  trigger  onset  times  and  does  not  have  the 
ability  to  encode  different  types  of  stimuli;  a  second  data  file  must  be  referenced  to  extract 
this  information. 

2.  Use  Third-Party  Synchronization  Software  at  Acquisition  Time.  Some  available  software 
packages  are  designed  specifically  for  recording  information  from  different  physical  and/or 
virtual  sources  (e.g.,  an  EEG  system,  motion  capture  device,  and  presentation  software). 
They  create  a  central  “clock”  to  which  all  other  activities  are  logged  in  reference,  so  that 
data  streams  are  inherently  synchronized,  even  when  data  sample  rates  may  be 
asynchronous.  Some  examples  are  Data  River  (Shwartz  Center  for  Computational 
Neuroscience,  University  of  California  at  San  Diego),  MAPPS  (EyesDX;  Coralville,  IA), 
and  BCI2000  (Wadsworth  Center  of  the  New  York  State  Department  of  Health  in  Albany, 
New  York).  By  using  a  central  clock  and  encoding  from  all  devices  at  run  time,  these 
dramatically  reduce  the  opportunities  for  drift-related  error.  Note,  however,  that  use  of 
these  software  systems  requires  them  to  already  support  all  devices/programs  a  researcher 
intends  to  use,  otherwise  the  scientist  must  work  with  the  developers  to  build  additional 
devices  into  the  data  stream. 

3.  Use  Embedded  Triggers  and  a  Synchronization  Algorithm  To  Align  Files  Off-Line.  This  is 
the  method  briefly  described  in  this  report.  In  summary,  a  series  of  trigger  tags  sent  from 
the  presentation  software  are  embedded  in  the  EEG  data  online  and  recorded  with  time 
stamps  in  a  separate  log  file,  along  with  any  supplemental  event  information.  Trigger 
onsets  between  the  two  files  (native  EEG  and  stimulus  file)  are  compared  and  synchronized 
using  regression  or  other  compensation  algorithms,  and  the  EEG-aligned  event  markers  are 
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modified  appropriately.  This  approach  requires  planning  at  least  the  synchronizing  pulses 
prior  to  run  time  and  assessing  the  system  drift.  However,  it  alleviates  the  need  for 
development  from  a  third-party  vendor  while  allowing  the  option  of  referring  to  data  not 
acquired  at  run  time.  Additionally,  we  have  shown  here  that  in  the  event  that  the  stimulus- 
data  recording  equipment  has  higher  fidelity  than  the  EEG  system,  this  method  can  even 
increase  the  overall  accuracy  of  the  reported  waveform  (as  evidenced  here  using  the 
Emotiv  Epoc  system). 
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